Scaling to a million cores and beyond: Using lightweight simulation to understand the challenges ahead on the road to exascale,” Future Generation Computer Systems (FGCS

As supercomputers scale to 1,000 PFlop/s over the next decade, investi-gating the performance of parallel applications at scale on future architec-tures and the performance impact of different architecture choices for high-performance computing (HPC) hardware/software co-design is crucial. This paper summarizes recent efforts in designing and implementing a novel HPC hardware/software co-design toolkit. The presented Extreme-scale Simulator (xSim) permits running an HPC application in a controlled environment with millions of concurrent execution threads while observing its performance in a simulated extreme-scale HPC system using architectural models and vir-tual timing. This paper demonstrates the capabilities and usefulness of the xSim performance investigation toolkit, such as its scalability to 227 simu-lated Message Passing Interface (MPI) ranks on 960 real processor cores, the capability to evaluate the performance of different MPI collective commu-nication algorithms, and the ability to evaluate the performance of a basic Monte Carlo application with different architectural parameters